Skip to content

GH-43702: [C++][FS][Azure] Use the latest Azurite and update the bundled Azure SDK for C++ to azure-identity_1.9.0 #43723

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
kou merged 2 commits into from
Aug 17, 2024
Merged

GH-43702: [C++][FS][Azure] Use the latest Azurite and update the bundled Azure SDK for C++ to azure-identity_1.9.0 #43723

kou merged 2 commits into from
Aug 17, 2024

Conversation

@kou
Copy link
Member

@kou kou commented Aug 16, 2024 •
edited
Loading

Rationale for this change

Some our CI jobs (such as conda based jobs) use recent Azure SDK for C++ and they require latest Azurite. We need to update Azurite for these jobs.

I wanted to use the latest Azurite on all environments but I didn't. Because I want to keep using apt install nodejs on old Ubuntu for easy to maintain.

What changes are included in this PR?

  • Use the latest Azurite if possible
  • Use --skipApiVersionCheck for old Azurite
  • Update the bundled Azure SDK for C++
    • This is not required. It's for detecting this problem in many CI jobs.

Are these changes tested?

Yes.

Are there any user-facing changes?

No.

@kou
Copy link
Member Author

kou commented Aug 16, 2024

@github-actions crossbow submit -g cpp -g python

@github-actions
Copy link

github-actions bot commented Aug 16, 2024

⚠️ GitHub issue #43702 has been automatically assigned in GitHub to PR creator.

@github-actions
Copy link

github-actions bot commented Aug 16, 2024

Revision: ee412c7

Submitted crossbow builds: ursacomputing/crossbow @ actions-de3559118a

Task Status
example-python-minimal-build-fedora-conda GitHub Actions
example-python-minimal-build-ubuntu-venv GitHub Actions
test-alpine-linux-cpp GitHub Actions
test-build-cpp-fuzz GitHub Actions
test-conda-cpp GitHub Actions
test-conda-cpp-valgrind GitHub Actions
test-conda-python-3.10 GitHub Actions
test-conda-python-3.10-cython2 GitHub Actions
test-conda-python-3.10-hdfs-2.9.2 GitHub Actions
test-conda-python-3.10-hdfs-3.2.1 GitHub Actions
test-conda-python-3.10-pandas-latest-numpy-1.26 GitHub Actions
test-conda-python-3.10-pandas-latest-numpy-latest GitHub Actions
test-conda-python-3.10-pandas-nightly-numpy-nightly GitHub Actions
test-conda-python-3.10-substrait GitHub Actions
test-conda-python-3.11 GitHub Actions
test-conda-python-3.11-dask-latest GitHub Actions
test-conda-python-3.11-dask-upstream_devel GitHub Actions
test-conda-python-3.11-hypothesis GitHub Actions
test-conda-python-3.11-pandas-upstream_devel-numpy-nightly GitHub Actions
test-conda-python-3.11-spark-master GitHub Actions
test-conda-python-3.12 GitHub Actions
test-conda-python-3.12-cpython-debug GitHub Actions
test-conda-python-3.8 GitHub Actions
test-conda-python-3.8-pandas-1.0-numpy-1.19 GitHub Actions
test-conda-python-3.9 GitHub Actions
test-conda-python-3.9-pandas-latest-numpy-latest GitHub Actions
test-conda-python-emscripten GitHub Actions
test-cuda-cpp GitHub Actions
test-cuda-python GitHub Actions
test-debian-12-cpp-amd64 GitHub Actions
test-debian-12-cpp-i386 GitHub Actions
test-debian-12-python-3-amd64 GitHub Actions
test-debian-12-python-3-i386 GitHub Actions
test-fedora-39-cpp GitHub Actions
test-fedora-39-python-3 GitHub Actions
test-ubuntu-20.04-cpp GitHub Actions
test-ubuntu-20.04-cpp-bundled GitHub Actions
test-ubuntu-20.04-cpp-minimal-with-formats GitHub Actions
test-ubuntu-20.04-cpp-thread-sanitizer GitHub Actions
test-ubuntu-20.04-python-3 GitHub Actions
test-ubuntu-22.04-cpp GitHub Actions
test-ubuntu-22.04-cpp-20 GitHub Actions
test-ubuntu-22.04-cpp-emscripten GitHub Actions
test-ubuntu-22.04-cpp-no-threading GitHub Actions
test-ubuntu-22.04-python-3 GitHub Actions
test-ubuntu-24.04-cpp GitHub Actions
test-ubuntu-24.04-cpp-gcc-13-bundled GitHub Actions
test-ubuntu-24.04-cpp-gcc-14 GitHub Actions

@kou
Copy link
Member Author

kou commented Aug 16, 2024

This also fixes #41505.

@kou
Copy link
Member Author

kou commented Aug 16, 2024

@github-actions crossbow submit -g python

@github-actions
Copy link

github-actions bot commented Aug 16, 2024

Revision: 3aa3bd6

Submitted crossbow builds: ursacomputing/crossbow @ actions-f0dea8bbe0

Task Status
example-python-minimal-build-fedora-conda GitHub Actions
example-python-minimal-build-ubuntu-venv GitHub Actions
test-conda-python-3.10 GitHub Actions
test-conda-python-3.10-cython2 GitHub Actions
test-conda-python-3.10-hdfs-2.9.2 GitHub Actions
test-conda-python-3.10-hdfs-3.2.1 GitHub Actions
test-conda-python-3.10-pandas-latest-numpy-1.26 GitHub Actions
test-conda-python-3.10-pandas-latest-numpy-latest GitHub Actions
test-conda-python-3.10-pandas-nightly-numpy-nightly GitHub Actions
test-conda-python-3.10-substrait GitHub Actions
test-conda-python-3.11 GitHub Actions
test-conda-python-3.11-dask-latest GitHub Actions
test-conda-python-3.11-dask-upstream_devel GitHub Actions
test-conda-python-3.11-hypothesis GitHub Actions
test-conda-python-3.11-pandas-upstream_devel-numpy-nightly GitHub Actions
test-conda-python-3.11-spark-master GitHub Actions
test-conda-python-3.12 GitHub Actions
test-conda-python-3.12-cpython-debug GitHub Actions
test-conda-python-3.8 GitHub Actions
test-conda-python-3.8-pandas-1.0-numpy-1.19 GitHub Actions
test-conda-python-3.9 GitHub Actions
test-conda-python-3.9-pandas-latest-numpy-latest GitHub Actions
test-conda-python-emscripten GitHub Actions
test-cuda-python GitHub Actions
test-debian-12-python-3-amd64 GitHub Actions
test-debian-12-python-3-i386 GitHub Actions
test-fedora-39-python-3 GitHub Actions
test-ubuntu-20.04-python-3 GitHub Actions
test-ubuntu-22.04-python-3 GitHub Actions

@github-actions github-actions bot added awaiting merge Awaiting merge and removed awaiting committer review Awaiting committer review labels Aug 16, 2024
@kou kou merged commit 8836535 into apache:main Aug 17, 2024
@kou kou removed the awaiting merge Awaiting merge label Aug 17, 2024
@kou kou deleted the cpp-azure-sdk-for-cpp branch August 17, 2024 02:20
@kou kou mentioned this pull request Aug 17, 2024
Closed
@conbench-apache-arrow
Copy link

conbench-apache-arrow bot commented Aug 17, 2024

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit 8836535.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@felipecrv felipecrv felipecrv approved these changes

@assignUser assignUser Awaiting requested review from assignUser assignUser is a code owner

@jonkeane jonkeane Awaiting requested review from jonkeane jonkeane is a code owner

@raulcd raulcd Awaiting requested review from raulcd raulcd is a code owner

Assignees

No one assigned

Labels

Component: C++ Component: Python

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[CI] Installing the newer version of Azurite is failing

2 participants

@kou @felipecrv